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ARI  Research  Reports  and  Technical  Papers  are  intended  for  sponsors  of 
R&D  tasks  and  other  research  and  military  agencies.  Any  findings  ready  for 
implementation  at  the  time  of  publication  are  presented  in  the  latter  part  of 
the  Brief.  Upon  completion  of  a major  phase  of  the  task,  formal  recommen 
dations  for  official  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 


FOREWORD 


A portion  of  the  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI)  program  consists  of  research  and  development  in 
areas  of  educational  technology  with  applicability  to  military  training. 
One  research  area  of  interest  is  the  use  of  alternative  training  media 
for  students  of  varying  abilities  and  characteristics.  Development  and 
implementation  of  such  systems  can  potentially  increase  the  efficiency 
and  effectiveness  of  the  training  process  because  the  instruction  can  be 
adapted  to  the  individual  student  to  a greater  extent  than  would  other- 
wise be  possible. 

A prerequisite  to  the  development  of  such  systems  is  the  identifi- 
cation of  those  student  characteristics  and  abilities  that  are  predictive 
of  training  success  with  specific  media.  Previous  research  had  suggested 
that  student  attitudes  toward  the  specific  medium  involved  were  potential 
predictors,  and  in  addition  had  produced  the  necessary  instruments  to 
measure  student  attitudes  toward  one  medium,  Computer-Assisted  Instruction 
(CAI) . The  purpose  of  this  research  was  to  investigate  more  extensively 
the  relationships  between  student  attitudes  toward  CAI  and  their  perfor- 
mance while  they  were  receiving  instruction  by  CAI. 

Whereas  this  effort  is  a basic  research  study,  it  initially  evolved 
because  of  and  in  support  of  an  operational  evaluation.  The  impetus  for 
this  project  was  a request  from  the  U.S.  Army  Ordnance  Center  and  School 
(USAOC&S)  for  scales  on  which  to  measure  student  attitudes  toward  CAI  to 
be  used  in  an  evaluation  of  the  training  effectiveness  of  the  PLATO  IV 
CAI  system.  To  meet  this  request,  basic  research  in  this  area  was  needed. 
A cooperative  effort  followed,  with  USAOC&S  receiving  the  scales  for 
their  use,  and  ARI  receiving  personnel  and  student  support  for  the  first 
phase  of  the  basic  research  project.  This  effort,  which  began  during 
FY  75,  was  part  of  the  Unit  Training  and  Educational  Technology  Systems 
Technical  Area  work  program.  Following  a reorganization,  the  effort 
continued  in  FY  76  as  part  of  the  Educational  Technology  and  Training 
Simulation  Technical  Area,  specifically  the  Educational  Concepts  and 
Evaluation  Work  Unit  area.  The  entire  research  effort  is  responsive  to 
FY  75  requirements  of  Project  2T161101A91B,  "Independent  Laboratory  In- 
House  Research . " 
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BRIEF 


Requirement : 

To  investigate  the  relationships  between  military  student  attitudes 
toward  Computer-Assisted  Instruction  (CAI)  and  performance  while  students 
were  receiving  instruction  by  CAI. 


Procedure : 

The  attitudes  of  90  military  enlisted  personnel  toward  CAI  were 
measured  before  and  after  they  received  a 107-minute  unit  of  instruction 
on  human  audition  presented  by  CAI.  Both  the  attitude  pretest  and  the 
attitude  posttest  were  administered  by  computer.  The  instructional  mate- 
rial was  divided  into  four  lesson  segments,  each  followed  by  a review 
(test)  segment.  Students  who  failed  to  reach  a specified  score  on  any 
review  segment  repeated  the  corresponding  lesson  segment.  A final  test, 
covering  the  material  from  all  segments,  was  administered  at  the  end  of 
the  last  segment.  Five  measures  of  student  performance  during  the  course 
of  the  instruction  were  used:  (a)  the  total  score  on  the  final  test 
(total  score) ; (b)  the  number  of  errors  made  during  the  lesson  and  review 
segments  (errors) ; (c)  the  percentage  of  responses  made  during  the  lesson 
and  review  segments  that  were  incorrect  (percent  errors) ; (d)  the  time 

required  to  complete  the  lesson  and  review  segments  (time) ; and  (e)  the 
number  of  review  segments  failed  (review  failures) . 


Findings : 

Scores  on  the  attitude  pretest  were  not  related  to  scores  on  any  of 
the  lesson  performance  criteria.  A significant  relationship  was  obtained 
between  the  attitude  posttest  percent  errors,  indicating  that  those  stu- 
dents who  made  a lower  percentage  of  errors  during  the  lesson  also  had 
more  positive  end-of-lesson  attitudes  toward  CAI.  Change  in  attitude  was 
related  to  total  score,  errors,  and  percent  errors.  Positive  change  in 
attitude  was  also  related  to  "good"  performance  on  these  criterion  mea- 
sures. Scores  on  both  attitude  scales  were  independent  of  student  abil- 
ity, as  measured  by  scores  on  the  AFQT  and  the  ACB. 


These  findings  indicate  that  student  attitudes  toward  CAI  are  of 
little  or  no  value  in  the  selection  of  training  media  and  methods  for 
individual  students,  at  least  when  the  students  have  had  limited  prior 
exposure  to  CAI. 


Utilization  of  Findings: 

The  on-line  versions  of  the  attitude  scales  are  available  for  use  by 
other  organizations  and  will  be  used  by  ARI  for  future  research. 
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RELATIONSHIPS  BETWEEN  STUDENT  ATTITUDES  TOWARD  COMPUTER-ASSISTED 
INSTRUCTION  AND  TRAINING  PERFORMANCE 


INTRODUCTION 

A central  theme  in  the  field  of  educational  technology  has  been  the 
development  and  evaluation  of  methods  that  allow  instruction  to  be  indi- 
vidualized. Both  training  specialists  and  educational  theorists  recog- 
nize the  importance  of  adapting  training  to  the  individual  student  if 
significant  advances  in  training  efficiency  or  effectiveness  are  to  be 
made . 


Training  can  be  individualized  in  many  ways.  Self-pacing,  either 
through  Programed  Instruction  (PI)  lessons  or  the  self-pacing  of  entire 
courses,  represents  perhaps  the  simplest  form  of  individualized  training. 
Computer-Assisted  Instruction  (CAI)  lessons  that  modify  the  amount  and 
sequence  of  instructional  material  on  the  basis  of  student  performance 
(or  student  choice)  represent  another  way  to  individualize  training.  A 
more  complex  method  is  to  assign  each  student  to  a particular  training 
method  or  medium  that  is  expected  to  be  most  effective  for  him  or  her. 

The  "best"  method  may  well  differ  for  each  student  as  a function  of  the 
type  of  instructional  material  presented.  There  are  three  mn  al 
requirements  for  implementing  individualized  training  of  the  last  type: 

(a)  multiple  methods  and  media  by  which  students  can  accomplish  the  same 
training  objectives;  (b)  decision  models  for  selecting  the  best  training 
methods  and  media  for  individual  students;  and  (c)  mechanisms  for  imple- 
menting those  decision  models.  Requirements  (a)  and  (c)  are  well  within 
the  scope  of  today's  technology.  However,  at  present  there  is  insuffi- 
cient knowledge  regarding  the  interactions  among  student  characteristics, 
instructional  treatments  (methods  and  media) , and  training  content  to 
permit  implementation  of  such  an  approach  to  individualized  training. 

Student  attitudes  toward  training  methods  and  media  constitute  one 
possible  basis  for  media  selection  decisions.  The  relatively  high  im- 
portance of  such  attitudes,  particularly  toward  CAI,  PI,  and  Computer- 
Managed  Instruction  (CMI) , as  perceived  by  both  Army  trainers  and 
researchers,  made  them  a logical  topic  for  further  investigation.  Army 
trainers  typically  assess  student  attitudes  toward  CAI  or  CMI  systems  as 
a part  of  the  evaluations  of  those  systems.  The  evaluations  of  the  PLATO 
IV  CAI  system  at  the  Army  Ordnance  Center  and  School  and  the  Computerized 
Training  Systems  Project  ABACUS  at  the  Army  Signal  School  are  recent  ex- 
amples. High  perceived  importance  by  researchers  is  evidenced  by  the 
substantial  number  of  studies  that  have  collected  student  attitude  data. 
King  (1975) , in  a review  of  the  research  literature  in  this  area,  cited 
44  such  studies . 


However,  this  importance  has  produced  neither  well- integrated  nor 
consistent  results,  particularly  with  regard  to  relationships  between 
student  attitudes  and  student  performance  in  the  instructional  situation. 
King  (1975)  noted  two  methodological  problems  which,  it  would  appear,  are 
at  least  partially  responsible  for  this  lack  of  integration  and  consis- 
tency. First,  although  student  attitude  data  are  frequently  collected  (as 
noted  above) , there  has  been  no  effort  to  investigate  student  attitudes  in 
any  integrated  fashion.  Only  rarely  are  student  attitudes  the  primary  fo- 
cus of  the  research.  Investigators  are  justifiably  more  concerned  with 
the  effects  of  the  independent  variables  on  cognitive  or  psychomotor  per- 
formance than  with  effects  on  student  attitudes.  Second,  there  is  little 
agreement  on  an  operational  definition  of  student  attitude.  There  have 
been  some  attempts  to  achieve  consistency  in  the  use  of  measuring  instru- 
ments: five  of  the  studies  cited  by  King,  and  one  additional  study  to  be 

cited  here  (Gallagher,  1970) , have  used  either  a scale  developed  by  Brown 
(1966)  or  modifications  thereof.  For  the  most  part,  however,  the  scales 
used  are  designed  ad  hoc  with  neither  the  items  used  nor  the  metric  prop- 
erties of  the  scales  described.  As  King  noted,  "Most  studies  use 
experimenter-constructed  tests  which  have  unknown  or  unreported 
reliabilities  (1975,  p.7)." 

Relatively  few  studies  have  attempted  to  investigate  what,  if  any, 
relationships  exist  between  attitudes  toward  PI,  CAI,  or  CMI  and  perfor- 
mance in  a course  of  instruction  taught  by  one  of  those  methods.  Studies 
that  have  attempted  to  find  relationships  have  produced  mixed  results. 

Doty  and  Doty  (1964)  found  posttest  attitude  (attitude  measured  following 
the  instruction)  toward  PI  to  be  positively  correlated  with  lesson 
achievement  for  female  students  but  not  for  male  students.  Gallagher 
(1970)  found  no  significant  relationships  between  posttest  attitude  toward 
CMI  and  course  performance.  Mathis,  Smith,  and  Hansen  (1970)  measured 
attitudes  toward  CAI  both  before  and  after  the  instruction  was  presented. 
Errors  per  question  attempted  during  the  course  of  the  instruction  corre- 
lated negatively  with  posttest  attitude  but  not  with  pretest  attitude. 
Reid,  Palmer,  Whitlock,  and  Jones  (1973)  measured  attitudes  toward  CAI 
before  and  after  the  instruction  was  presented.  Pretest  attitude,  post- 
test attitude,  and  attitude  change  were  unrelated  to  lesson  achievement 
and  to  the  amount  of  time  required  to  complete  the  lesson  for  the  total 
sample.  Although  significant  correlations  between  attitudes  and  perfor- 
mance were  obtained  for  selected  subgroups,  it  appears  that  no  more  than 
5.2%  of  the  correlations  calculated  were  significant  at  the  .05  level; 
thus  these  correlations  could  be  attributed  to  chance  alone. 

The  above  studies  were  conducted  using  college  or  graduate  students, 
and  the  attitude  scales  used  were  not  felt  to  be  appropriate  for  use  with 
enlisted  military  personnel.  Consequently,  Knerr  and  Nawrocki  (1978) 
developed  two  attitude- toward-CAI  scales,  a 13-item  pretest  and  a 37-item 
posttest,  for  use  with  military  personnel.  When  used  in  the  evaluation 
of  the  PLATO  IV  CAI  system  at  the  U.S.  Army  Ordnance  Center  and  School, 
neither  the  pretest  nor  the  posttest  was  correlated  with  the  time  required 
to  complete  a series  of  four  lessons  presented  by  CAI,  the  only  perfor- 
mance criterion  available. 
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The  purpose  of  this  study  was  to  continue  the  investigation  of  the 
relationships  between  military  student  attitudes  toward  CAX  and  student 
performance  begun  by  Knerr  and  Nawrocki  (1978) , using  a variety  of  mea- 
sures of  performance  in  the  instructional  situation. 


METHOD 


Subjects 


Subjects  were  90  enlisted  military  personnel  receiving  training  at 
the  U.S.  Army  Engineer  School  during  the  period  September  to  November 
1975.  A description  of  salient  student  background  information  is  pro- 
vided in  Table  1.  Median  values  show  that  the  typical  student  was  male, 
was  in  pay  grade  E-2,  was  19  years  old,  had  been  in  the  military  for  3 
months,  had  12  years  of  formal  education  (all  were  at  least  high  school 
graduates) , and  could  not  type. 


Table  1 

Student  Background  Information 


Item 

Median 

Range 

Pay  grade 

E-2 

El  - E5 

Age 

19 

17  - 32 

Months  in  service 

3 

2-53 

Years  education 

12 

12  - 16 

Typing  speed 

(words  per  minute) 

0 

0 - 100 

Male 

Female 

Sex 

81 

7 

Note. 


N = 88.  Background  data  on  two  students  were  destroyed  by 
systems  failure. 
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Attitude  Scales 


The  two  attitude  scales,  pretest  and  posttest,  developed  by  Knerr  and 
i Nawrocki  (1978)  were  used.  The  pretest  consists  of  13  five-alternative 

multiple  choice  items.  When  previously  administered,  it  was  found  to  have 
an  internal  consistency  (Kuder-Richardson  Formula  8;  KR  8)  of  .91.^  The 

posttest  consists  of  37  items  and  has  an  internal  consistency  of  .89;  it 

can  be  divided  into  two  subtests.  The  first,  the  repeated  subtest,  con- 
sists of  those  13  items  in  the  pretest  (rephrased  in  the  past  tense  where 

necessary)  and  has  an  internal  consistency  of  .85.  The  unique  subtest 
consists  of  24  additional  items  and  has  an  internal  consistency  of  .81. 

The  pretest  and  posttest  items  are  shown  in  appendixes  A and  B 
respectively. 

The  scales  had  previously  been  administered  in  paper  and  pencil  for- 
mat. Prior  to  this  study  they  were  modified  for  on-line  administration 
and  scoring  by  the  PLATO  IV  CAI  system. 2 The  experimental  procedure  was 
such  that  the  scales  had  to  be  administered  on  an  individual  basis.  Com- 
puterized administration,  then,  offered  a substantial  time  savings  for 
the  experimenter.  It  also  provided  the  capability  for  automatic  scoring 
and  data  reduction. 

Figure  1 shows  a sample  item  as  it  appeared  initially  to  the  student. 
The  student  responded  by  entering  the  letter  of  his  or  her  choice  (a 
through  e)  via  the  terminal  keyboard.  This  choice  was  then  marked  by  an 
asterisk,  and  the  student  was  given  an  opportunity  to  change  the  answer 
before  advancing  to  the  next  question,  as  shown  in  Figure  2.  The  student 
was  required  to  answer  each  question  in  order  to  advance  to  the  next . 


This  measure  entails  fewer  assumptions  than  the  more  commonly  used  Kuder- 
Richardson  formula  20  (KR  20) . Specifically,  KR  8 assumes  that  the  inter- 
correlation matrix  has  a rank  of  one,  that  is,  that  the  scale  measures 
only  one  factor,  while  KR  20  assumes,  in  addition,  that  all  item  intercor- 
relations and  standard  deviations  are  equal.  If  these  additional  assump- 
tions are  met,  KR  8 and  KR  20  will  produce  identical  reliability  estimates. 
If  they  are  not  met,  KR  8 will  produce  higher  and  more  accurate  reliability 
estimates  than  KR  20. 

2 

Commercial  designations  are  used  only  for  precision  of  description. 

Their  use  does  not  constitute  endorsement  by  the  Army  or  the  Army  Research 
Institute . 
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I 


Most  courses  could  be  taught  more  effectively  by 

a regular  teacher 

than  by  computer. 

a . 

Strongly  agree 

b. 

Agree 

c . 

Undecided 

d. 

Disagree 

e . 

Strongly  disagree 

Figure  1.  Pretest  item  1,  as  it  appears  to  the  student 
before  a response  is  selected. 


Most  courses  could  be  taught  more  effectively  by 
a regular  teacher  than  by  computer. 

a.  Strongly  agree 
* b.  Agree 

c.  Undecided 

d.  Disagree 

e.  Strongly  disagree 

Your  answer  is  marked  with  an  ' * ' . You  can  changi 
it  by  pressing  -BACK-,  or  continue  with  the  next 
question  by  pressing  -NEXT-. 


Figure  2.  Pretest  item  1,  as  it  appears  to  the  student 
after  a response  is  selected. 


Lesson  Material 


The  lesson  material  consisted  of  a four-section  lesson  entitled  "The 
Elementary  Physiology  of  Audition  (or  How  Your  Ear  Works)."  The  sections 
dealt  with  sound  waves,  anatomy  of  the  ear,  balance,  and  ear  damage  and 
degeneration,  respectively.  Following  each  section,  the  students  received 
a series  of  review  questions  covering  the  objectives  of  that  section. 
Students  who  answered  less  than  85%  of  the  review  questions  for  any  sec- 
tion correctly  were  required  to  repeat  that  section.  After  the  fourth 
review  was  successfully  completed,  the  student  was  administered  a 68-item 
test  covering  the  objectives  of  the  entire  lesson. 


Performance  Criteria 


The  following  performance  criteria  were  obtained  for  each  student: 

(a)  the  amount  of  time  required  to  complete  the  four  lesson  and  review 
sections  (time) ; (b)  the  total  score  on  the  final  end-of- lesson  test 

(total  score) ; (c)  the  number  of  incorrect  responses  made  during  the 
lesson  and  review  sections  (errors) ; (d)  the  percentage  of  the  total 
responses  made  during  the  lesson  and  review  sections  that  were  incorrect 
(percent  errors) ; and  (e)  the  number  of  reviews  failed  (review  failures) . 
Percent  errors  requires  some  additional  explanation.  It  is  the  percentage 
of  responses  to  which  the  student  did  not  receive  positive  feedback  (OK, 
correct,  right,  etc.).  It  is  also  similar  to  errors  per  question  at- 
tempted, which  Mathis  et  al.  (1970)  found  to  be  correlated  with  posttest 
attitude . ^ 


Experimental  Procedure 

Upon  arrival  at  the  experimental  area,  each  student  was  given  a short 
written  explanation  of  the  experimental  procedures.  After  any  questions 
were  answered  he  or  she  was  signed  onto  the  PLATO  IV  system.  The  PLATO 
IV  system  controlled  all  student  routing,  lesson  administration,  and  data 
collection.  Unless  a problem  arose,  the  student  did  not  interact  with  the 
experimenter  again  until  the  end  of  the  session. 

After  being  signed  onto  the  system,  each  student  received  a short 
lesson  on  the  use  of  the  keyboard  and  terminal.  This  was  followed  by  an 
on-line  biographic  ques  .ionnaire , the  attitude  pretest,  the  human  audition 
lesson,  the  end-of- lesson  test,  and  the  attitude  posttest. 


3 

It  is  not  clear  whether  Mathis  et  al.  (1970)  used  errors  per  attempt  or 
errors  per  question  as  their  performance  measure.  In  CAI  lessons,  in 
general,  a student  may  attempt  questions  more  than  once;  the  two  measures, 
therefore,  are  not  necessarily  equal.  Percent  errors  is  equal  to  errors 
per  attempt,  and  a monotonic  (but  nonlinear)  function  of  errors  per 
question. 
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Selection  and  Classification  Test  Scores 


lk.1 


After  the  experiment  had  been  completed,  scores  on  the  Armed  Forces 
Qualification  Test  (AFQT)  and  the  General  Technical  (GT)  aptitude  area  of 
the  Army  Classification  Battery  were  obtained  for  approximately  one-half 
of  the  students.  These  scores  permitted  examination  of  the  relationships 
between  "ability"  and  attitude. 


RESULTS 

The  results  are  organized  around  five  separate  issues.  The  first 
two  are  concerned  with  the  characteristics  of  the  attitude  scales  them- 
selves: scale  administration  time  and  internal  consistency.  The  second 

two  deal  with  the  relationships  between  the  attitude  scale  scores  and  the 
performance  criteria,  and  the  relationships  between  attitude  change  and 
the  performance  criteria,  respectively.  The  final  issue  is  the  relation- 
ships among  student  ability,  student  performance,  and  student  attitude. 


Attitude  Scale  Administration  Time 


Although  the  amount  of  time  required  for  the  students  to  complete 
the  attitude  scales  is  not  directly  related  to  the  primary  objective  of 
this  research,  it  does  provide  some  information  about  the  feasibility  of 
on-line  administration.  Extremely  long  administration  times  would  be 
indicative  of  student  problems  and  would  also  suggest  that  administration 
in  an  operational  setting  is  not  feasible.  Table  2 shows  the  administra- 
tion time  data  for  87  students.  (A  systems  error  caused  the  loss  of  time 
data  for  the  other  three  students.)  Mean  administration  time  for  the  pre- 
test is  4.90  minutes,  with  90%  of  the  students  completing  it  in  6.90  min- 
utes or  less.  Mean  administration  time  for  the  posttest  is  9.17  minutes, 
with  90%  completing  it  in  11.80  minutes  or  less. 


Table  2 


Attitude  Scale 

Administration  Time 

(in  Minutes) 

Time 

Pretest 

Posttest 

Mean 

4.90 

9.17 

Median 

4.40 

8.75 

Standard  deviation 

1.81 

2.32 

Note.  N = 87.  Data  on  three  students  were  lost. 
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Internal  Consistency 

The  item  stem,  mean,  standard  deviation,  and  item-total  correlation 
for  each  item  are  included  in  appendixes  A (pretest)  and  B (posttest) . 
Table  3 presents  a comparison  of  the  internal  consistencies  (KR  8)  of  the 
on-line  versions  of  the  pretest,  total  posttest,  and  unique  and  repeated 
subtests  with  the  internal  consistencies  of  the  previously  administered 
off-line  (paper  and  pencil)  versions.  The  internal  consistencies  for  the 
on-line  versions  are  .88  for  the  pretest,  .93  for  the  repeated  posttest, 
.87  for  the  unique  posttest,  and  .93  for  the  total  posttest.  Each  of 
these  values  was  compared  with  the  internal  consistency  of  the  same  scale 
when  administered  in  paper  and  pencil  format.  Differences  were  tested, 
using  the  z test  for  the  significance  of  the  difference  between  two  corre- 
lations. Table  3 shows  the  results  of  these  tests.  Whereas  it  initially 
appeared  that  the  on-line  versions  of  the  posttest  had  higher  internal 
consistencies  than  did  the  off-line  versions,  this  is  supported  only  for 
the  repeated  posttest,  for  which  the  on-line  version  has  an  internal  con- 
sistency of  .93,  and  the  off-line  version,  .85  (z  = -2.40,  p < .01). 


Table  3 


A Comparison  of  the  Internal  Consistencies  (KR  8) 
of  the  On-Line  and  Off-Line  Versions  of  the  Attitude  Scales 


Tests 

Off  line 

. a 

On  line 

z 

difference 

Pretest 

.91b 

.88 

1.20 

Repeated  posttest 

. 85° 

.93 

-2.40* 

Unique  posttest 

.81C 

.87 

-1.20 

Total  posttest 

.89° 

.93 

-1.44 

aN  = 90. 


bN  = 228. 
CN  = 64. 

*p  < .01. 
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Attitude-Performance  Relationships 


The  matrix  of  intercorrelations  among  the  attitude  scale  scores  and 
the  lesson  performance  criteria  is  shown  in  Table  4.  The  left  side  of 
the  table  indicates  that  the  performance  criteria  are  highly  correlated, 
with  the  absolute  value  of  the  correlations  ranging  from  .57  to  .90. 

High  values  of  total  score  reflect  good  performance,  whereas  high  values 
of  the  other  performance  criteria  reflect  poor  performance.  Thus  the 
correlations  between  total  score  and  the  other  performance  criteria  are 
negative . 

The  lower  right  corner  of  the  intercorrelation  matrix  shows  the 
intercorrelations  among  the  various  attitude  scales.  Correlations  of  the 
pretest  with  the  repeated,  unique,  and  total  posttests  are  .78,  .59,  and 
.71,  respectively.  The  correlation  between  the  unique  and  repeated  post- 
tests is  .78. 

The  attitude  pretest  appears  to  have  no  predictive  validity.  Corre- 
lations of  the  pretest  with  the  performance  criteria  range  from  -.02  to 
.09.  None  of  these  correlations  is  significant  at  the  .05  level. 

Two  of  the  performance  criteria  show  significant  relationships  with 
the  postlesson  attitude  scales.  Total  score  is  correlated  with  the  unique 
posttest  (r  = .21,  p < .05).  Percent  errors  is  correlated  with  the 
unique  posttest  (r  = -.22,  p < .05)  and  the  total  posttest  (r  = -.22, 
p < .05).  The  square  of  these  correlations  indicates  that  between  4.41% 
and  4.84%  of  the  variance  in  posttest  attitude  is  accounted  for  by  the 
performance  criteria. 


Attitude  Change  and  Performance 

The  relationships  between  attitude  change  and  performance  criteria 
were  also  determined.  The  measure  of  attitude  change  used  was  residual 
change,  that  is,  the  portion  of  the  posttest  score  that  is  independent 
of  the  pretest  score.  Use  of  residual  change  ir>  this  situation  has  two 
advantages  over  the  use  of  a raw  gain  score  (posttest  minus  pretest) : 

(a)  it  is  not  correlated  with  pretest  score,  while  raw  gain  is  negatively 
correlated  with  pretest  score;  and  (b)  it  does  not  require  that  the  pre- 
test and  posttest  scores  have  the  same  metric  (Manning  & DuBois,  1962). 
Correlations  between  the  performance  criteria  and  residual  change  were 
calculated  by  using  the  part  correlation  (McNemar,  1962).  Thus  the  corre- 
lation calculated  is  that  between  a performance  criterion  and  a posttest 
score  with  the  effects  of  the  pretest  removed  from  the  posttest  score. 
Three  different  residual  changes  calculated  were  the  following:  pretest- 
unique  posttest,  pretest-repeated  posttest,  and  pretest- total  posttest. 


Correlations  Between  the  Attitude  Scales  and  the  Lesson  Performance  Criteria 


The  resulting  correlations  are  presented  in  Table  5.  Time  and 
review  failures  are  not  correlated  significantly  with  any  of  the  resid- 
uals. Total  score  and  percent  errors  are  correlated  with  each  of  the 
residuals.  Errors  are  correlated  only  with  the  pretotal  residual.  The 
absolute  value  of  these  significant  correlations  ranges  from  .21  (errors 
and  the  pretotal  residual)  to  .28  (total  score  and  the  pretotal  and  pre- 
unique residuals) . Thus  the  proportion  of  variance  in  attitude  change 
accounted  for  by  the  performance  criteria,  when  significant,  varies  be- 
tween 4.41%  and  7.84%. 


Table  5 


Correlations  Between  Residual  Attitude  Change 
and  the  Lesson  Performance  Criteria 


Performance 

criteria 

N 

Type  of  residual 

Pretotal 

Prerepeated 

Preunique 

Total  score 

90 

.28** 

.22* 

.28** 

Errors 

87 

-.21* 

-.18 

-.20 

Percent  errors 

87 

-.25* 

-.22* 

-.23* 

Time 

90 

-.12 

-.08 

-.13 

Review  failures 

89 

-.03 

.07 

-.09 

*p  < .05. 

**p  < .01. 


Student  Ability,  Attitude,  and  Performance 

GT  and  AFQT  scores  were  available  for  42  students.  These  scores 
were  correlated  with  the  performance  criteria  and  the  attitude  scale 
scores,  with  the  results  shown  in  Table  6.  Because  the  GT  and  AFQT  scores 
are  highly  correlated  (r  (41)  = .88,  p < .01),  they  show  similar  correla- 
tions with  the  other  variables.  Both  are  correlated  significantly  with 
each  of  the  lesson  performance  criteria.  In  all  cases,  students  with 
higher  AFQT  and  GT  scores  obtained  better  scores  on  the  performance 
criteria . 


Table  6 


Correlations  of  GT  and  AFQT  Scores  With  the  Attitude 
Scale  Scores  and  the  Lesson  Performance  Criteria 


Measure 

GT 

AFQT 

Total  score 

.66* 

.67* 

Errors 

-.61* 

-.54* 

Percent  errors 

-.40* 

-.42* 

Time 

-.77* 

-.79* 

Review  failures 

-.65* 

-.66* 

Pretest 

-.20 

-.06 

Repeated  posttest 

-.16 

-.02 

Unique  posttest 

.04 

.16 

Total  posttest 

-.06 

.08 

N = 42 

Mean 

111.79 

66.14 

Standard  deviation 

12.60 

18.20 

*p  < .01. 


In  contrast,  GT  and  AFQT  scores  are  not  correlated  with  any  of  the 
attitude  scale  scores.  Also,  as  shown  in  Table  7,  GT  and  AFQT  scores  are 
not  correlated  with  attitude  change.  In  summary,  total  score,  errors, 
and  percent  errors  are  related  to  attitude  change;  GT  and  AFQT  scores  are 
related  to  total  score,  errors,  and  percent  errors;  but  GT  and  AFQT 
scores  are  not  related  to  attitude  change.  A possible  explanation  for 
this  is  that  the  relationships  between  attitude  change  and  performance 
are  too  small  to  contribute  to  relationships  between  "ability"  and  atti- 
tude change . 
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Table  7 


Correlations  Between  Residual  Attitude  Change  and  GT  and  AFQT  Scores 


Test 

Pretotal 

Prerepeated 

Preunique 

GT 

.10 

-.02 

.17 

AFQT 

.17 

+ .04 

.23 

N = 42 

DISCUSSION 

Based  on  the 
student  attitudes 

results  of 

toward  CAI 

this  study , it  appears 
through  the  use  of  on- 

feasible  to  measure 

line  attitude  scales 

Certainly  there  are  many  questions  that  could  be  raised  about  such  an 
approach.  One  such  question  is  whether  the  method  of  administration  (on- 
line versus  paper  and  pencil)  produces  differential  student  responses.  A 
related  question  is  whether  the  method  of  administration  has  any  effect 
on  the  metric  properties  of  the  scales.  There  was  one  significant  dif- 
ference between  the  internal  consistencies  of  the  on-line  versions  and 
those  of  Knerr  and  Nawrocki's  (1978)  paper-and-pencil  versions;  however, 
the  background  characteristics  of  the  students  in  the  two  studies  were 
not  exactly  equivalent  and,  more  importantly,  the  amount  of  exposure  that 
the  students  had  to  CAI,  both  before  the  administration  of  the  pretest 
and  between  the  administration  of  the  pretest  and  the  posttest,  differed 
for  the  two  groups.  Thus  the  difference  obtained  may  be  a result  of  fac- 
tors other  than  the  method  of  administration. 

Despite  these  uncertainties,  on-line  administration  appears  to  have 
certain  advantages.  The  first  advantage  is  the  reduction  of  personnel 
time  required  for  administration  and  scoring;  a second  is  that  the  scales 
can  be  integrated  into  the  computer  system's  management  function,  thus 
insuring  that  each  student  is  administered  the  proper  scale  at  the  proper 
time;  finally,  the  method  of  presentation  eliminates  certain  types  of 
student  errors,  such  as  answering  a question  in  the  wrong  space  on  an 
answer  sheet,  or  failing  to  answer  a question. 
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The  internal  consistencies  of  .88  for  the  pretest  and  .93  for  the 
total  posttest  compare  favorably  with  those  of  previously  developed 
scales.4  Few  other  scales  that  could  be  used  as  pretests  have  reported 
reliabilities.  Gallagher  (1970)  reported  an  internal  consistency  of  .937 
(Hoyt  method)  for  the  Michigan  State  University  Learning  Service  "Attitude 
Toward  Learning  Inventory."  This  scale  is  suitable  for  use  as  a pretest, 
but  it  is  not  designed  specifically  to  measure  attitudes  toward  CAI . 

Mathis  et  al.  (1970)  developed  a 30-item  pretest  but  reported  no  internal 
consistency  for  it. 

Internal  consistencies  (KR  20  or  equivalent)  for  other  scales  used 
as  posttests  are  .89  for  Brown's  (1966)  40-item  scale  and  .82  for  the  27- 
item  scale  developed  by  Mathis  et  al.  (1970).  Many  scales,  such  as  those 
used  by  Doty  and  Doty  (1964)  and  Reid  et  al.  (1973),  have  no  reported 
internal  consistencies. 

Evidence  is  mounting  that  preinstructional  attitude  toward  CAI  does 
not  affect  subsequent  performance  in  the  instructional  situation,  at 
least  when  relatively  naive  students  are  involved.  Knerr  and  Nawrocki, 
(1978),  using  time  as  the  criterion,  found  this  to  be  the  case.  Mathis 
et  al.  (1970)  obtained  similar  results,  using  errors  per  question 
attempted  as  the  criterion.  In  the  present  study,  no  predictive  relation- 
ships were  found,  although  a variety  of  lesson  performance  criteria  were 
used. 


These  results  may  not  be  generalizable  to  students  who  have  had  ex- 
tensive previous  experience  with  CAI.  Mathis  et  al.  (1970)  and  Knerr  and 
Nawrocki  (1978)  used  as  subjects  students  who  had  no  previous  experience 
with  CAI.  In  the  current  study,  the  experience  of  the  students  prior  to 
the  administration  of  the  pretest  was  limited  to  the  time  required  to 
complete  the  terminal  familiarization  lesson  and  the  biographic  informa- 
tion questionnaire — less  than  30  minutes.  Thus  CAI  was  still  a novel  and 
somewhat  unfamiliar  situation  for  the  students.  It  is  not  known  whether 
similar  results  would  have  been  obtained  if  the  students  had  previously 
received  instruction  by  CAI. 

Certain  aspects  of  student  performance  in  the  instructional  situation 
are  related  to  posttest  attitude  (total  score  and  percent  errors)  and  to 
attitude  change  during  the  course  of  instruction  (total  score,  errors,  and 
percent  errors) . These  effects,  however,  were  not  large.  None  of  the 


Other  internal  consistencies  reported  have  been  calculated  using  formula 
KR  20  or  an  algebraic  equivalent.  For  the  scales  used  in  this  study,  KR 
20  internal  consistencies  were  .87  for  the  pretest  and  .92  for  the  post- 
test. The  KR  20  values  are  less  accurate  than  the  KR  8 values,  but  they 
provide  a fairer  comparison  with  other  scales. 
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performance  criteria  accounted  for  more  than  5%  of  the  variance  in  post- 
test attitude  or  more  than  8%  of  the  variance  in  attitude  change. 


Student  ability,  as  measured  by  AFQT  and  GT  scores,  was  related  to 
neither  attitude  scale  scores  nor  attitude  change.  This  result  provides 
some  evidence  for  the  construct  validity  of  the  attitude  scales.  At  a 
minimum,  it  shows  that  the  attitude  scales  measure  something  other  than 
student  ability.  This  is  a necessary  requirement  for  the  establishment 
of  construct  validity,  but  it  is  not  a sufficient  one.  According  to  the 
APA  Standards  for  Educational  and  Psychological  Tests, 

Evidence  of  construct  validity  is  not  found  in  a 
single  study;  rather,  judgments  of  construct 
validity  are  based  upon  an  accumulation  of  research 
results  (1974,  p.  30). 

Future  research  examining  the  relationships  between  the  attitude  scale 
scores  and  other  variables  would  be  required  to  establish  the  construct 
validity  of  the  scales. 


CONCLUSIONS 

In  view  of  the  lack  of  predictive  relationships  between  the  attitude 
pretest  and  subsequent  performance,  measuring  pretest  attitude  for  naive 
students  would  seem  to  be  of  little  value  unless  one  is  interested  in 
studying  attitude  change,  per  se.  Student  attitudes  do  not  appear  to  be 
useful  for  deciding  whether  a particular  student  should  receive  instruc- 
tion by  CAI  or  by  some  other  medium.  This  may  not  be  the  case  for  stu- 
dents who  have  had  extensive  previous  experience  with  CAI,  however. 

Preparation  of  CAI  lesson  materials  that  produce  low  student  error 
rates  during  the  course  of  instruction  and  on  the  postlesson  test  may  be 
beneficial  in  terms  of  maintaining  or  producing  positive  student  atti- 
tudes toward  CAI.  It  has  been  demonstrated  that  certain  aspects  of  stu- 
dent performance  are  related  to  subsequent  student  attitudes;  neverthe- 
less, a causal  relationship  has  yet  to  be  established. 
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APPENDIX  A 
PRETEST  ITEM  SUMMARY 


' 


This  appendix  includes  a list  of  all  pretest  items,  with  the  mean, 
standard  deviation  (s.d.),  and  item-total  correlation  (r^)  for  each. 

All  odd-numbered  items  are  negatively  worded,  that  is,  a "strongly  agree, 
"all  of  it,"  "all  the  time,"  or  "quite  often"  response  is  the  one  least 
favorable  toward  CAI.  All  even-numbered  items  are  positively  worded. 

Response  alternatives  are  coded  as  follows: 


A - a.  Strongly  agree 

b.  Agree 

c.  Undecided 

d.  Disagree 

e.  Strongly  disagree 

P - a.  All  of  it 

b.  75% 

c.  50% 

d.  25% 

e.  None  of  it 


T 


a.  All  the  time 

b.  Most  of  the  time 

c . Some  of  the  time 

d.  Only  occasionally 

e . Never 


F - a.  Quite  often 

b.  Often 

c.  Occasionally 

d.  Seldom 

e.  Very  seldom 


All  data  are  based  on  a sample  size  of  90. 
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Item 
no . 


Item  stem 


Mean 


s . d . 


1 


1.  Most  courses  could  be  taught  more 
effectively  by  a regular  teacher 
than  by  computer.  (A) 

2.  I would  like  to  take  a lesson 
taught  by  CAI.  (A) 

3.  Taking  a lesson  taught  by  CAI 
would  make  me  nervous.  (A) 

4.  Taking  a course  taught  by  computer 
would  be  more  interesting  than 
taking  the  same  course  taught  in 
some  other  way.  (A) 

5.  People  should  be  taught  by  other 
people,  not  by  machines.  (A) 

6 . I would  prefer  to  have  most  courses 
taught  by  computer  rather  than  by 
other  teaching  methods.  (A) 

7.  I think  I would  feel  isolated  and 
alone  while  taking  a course  taught 
by  computer.  (A) 

8.  I think  I would  feel  challenged  to 
do  my  best  work  while  taking  a 
course  taught  by  computer.  (T) 

9.  It  would  be  boring  to  take  a course 
taught  by  computer.  (A) 

10.  I think  it  would  be  easy  to  under- 
stand the  material  in  a course 
taught  by  computer.  (A) 

11.  Students  are  being  treated  more 
and  more  like  IBM  cards.  (A) 

12.  How  much  of  this  lesson  do  you 
think  you  would  like  to  have 
taught  by  computer?  (P) 

13.  Taking  a lesson  taught  by  computer 
would  be  too  mechanical.  (A) 


2.96  0.92 

4.21  0.91 

3.89  0.98 

3.60  1.14 

3.13  1.05 

2.97  1.09 

3.51  1.13 

3.81  1.04 

3.53  1.10 

3.49  0.81 

2.99  1.34 

4.28  1.08 

3.54  1.06 


.47 

.60 

.37 


.66 


.62 


.74 


.67 


.62 


.64 


.54 


.61 


.62 

. 77 


_ 
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APPENDIX  B 

POSTTEST  ITEM  SUMMARY 


This  appendix  includes  a list  of  all  posttest  items,  with  the  mean, 
standard  deviation  (s.d.),  and  item-total  correlation  (r^t)  for  each.  All 
odd-numbered  items  are  negatively  worded,  that  is,  a "strongly  agree," 

"all  of  it,"  "all  the  time,"  or  "quite  often"  response  is  the  one  least 
favorable  toward  CAI . All  even-numbered  items  are  positively  worded. 

Response  alternatives  are  coded  as  follows: 

A - a.  Strongly  agree 

b.  Agree 

c.  Undecided 

d.  Disagree 

e.  Strongly  disagree 

P-a.  All  of  it 

b.  75% 

c . 50% 

d.  25% 

e.  None  of  it 

T - a.  All  the  time 

b.  Most  of  the  time 

c.  Some  of  the  time 

d.  Only  occasionally 

e.  Never 

F - a.  Quite  often 

b.  Often 

c.  Occasionally 

d.  Seldom 

e.  Very  seldom 

All  data  are  based  on  a sample  size  of  90. 


no.  Item  stem 

1.  The  way  the  material  was  presented 
to  me  made  me  feel  that  no  one 
really  cared  whether  I learned  or 
not.  (A) 

2.  Based  on  my  experience  with  this 
lesson,  I prefer  CAI  to  other  meth- 
ods of  instruction.  (A) 

3.  The  method  by  which  I was  told 
whether  I had  given  a right  or 
wrong  answer  became  boring.  (A) 

4.  I felt  as  if  someone  were  engaged 
in  conversation  with  me.  (T) 

5.  I was  concerned  that  I might  not 
be  understanding  the  material.  (T) 

6.  The  responses  to  my  answers  were 
appropriate.  (T) 

7.  I felt  uncertain  as  to  my  perfor- 
mance compared  to  the  performance 
of  others.  (T) 

8.  I knew  whether  my  answers  were 
correct  or  not  before  I was  told. 

(F) 

9.  I found  myself  just  trying  to  get 
through  the  lesson  rather  than 
trying  to  learn.  (T) 

10.  I was  encouraged  by  the  responses 
given  to  my  answers.  (T) 

11.  I guessed  at  the  answers  to 
questions.  (F) 

12.  I was  able  to  work  at  my  own 
pace.  (T) 

13.  In  view  of  the  time  allowed,  I 
felt  too  much  material  was  pre- 
sented. (T) 


3.77 


1.40 


.53 


14.  I Celt  as  if  I had  a private 
instructor.  (T) 


15. 

i 

16. 

(- 

17. 

18. 

19. 

20. 

21. 

22. 

23. 

24. 

25. 


I was  more  involved  in  running  the 
machine  than  in  understanding  the 
material.  (T) 

I was  aware  of  efforts  to  suit  the 
material  specifically  to  me.  (F) 

I found  it  difficult  to  concen- 
trate on  the  course  material  because 
of  the  machine.  (T) 

Computer-assisted  instruction  made 
it  possible  for  me  to  learn 
quickly.  (A) 

Questions  were  asked  which  were 
not  relevant  to  the  material 
presented.  (T) 

The  responses  to  my  answers  seemed 
to  take  into  account  the  difficulty 
of  the  question.  (T) 

In  order  to  get  more  information 
from  the  machine,  I gave  answers 
which  I knew  were  wrong.  (F) 

In  view  of  the  effort  I put  into  it, 
I was  satisfied  with  what  I learned 
while  taking  CAI.  (A) 

.1  was  given  answers  but  still  did 
not  understand  the  questions.  (F) 

In  view  of  the  amount  I learned,  I 
would  say  that  CAI  is  superior  to 
other  teaching  methods.  (A) 

Most  courses  could  be  taught  more 
effectively  by  a regular  teacher 
than  by  computer.  (A) 


4.11 

3.24 

4.24 

3.92 

4.67 

3.04 

4.51 

4.34 

4.17 

3.44 

3.12 


26.  I would  like  to  take  another  lesson 

which  uses  CAI.  (A)  4.13 


27.  Taking  a lesson  taught  by  CAI  made 

me  nervous.  (A)  4.10 


0.95 

1.33 

0.95 

1.05 

0.67 

1.23 

0.92 

0.85 

0.96 

1.06 

1.22 

1.10 

1.08 


.52 

.49 

.50 

.64 

.11 

.28 

.29 

.65 

.60 

.64 

.67 

. 73 

.34 
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28.  The  material  taught  by  computer 
was  more  interesting  than  taking 
similar  material  taught  in  some 
other  way.  (A) 

29.  People  should  be  taught  by  other 
people,  not  by  machines.  (A) 

30.  I would  prefer  to  have  most  courses 
taught  by  computer  rather  than  by 
other  teaching  methods.  (A) 

31.  I felt  isolated  and  alone  while 
working  with  the  computer.  (A) 

32.  I felt  challenged  to  do  my  best 
work  while  being  taught  by 
computer.  (T) 

33.  It  was  boring  to  learn  material 
taught  by  computer.  (A) 

34.  It  was  easy  to  understand  the 
material  taught  by  computer.  (A) 

35.  Students  are  being  treated  more  and 
more  like  IBM  cards.  (A) 

36.  How  much  of  the  lesson  you  just 
completed  do  you  think  should  be 
taught  by  computer?  (P) 

37.  The  lessons  taught  by  computer 
were  too  mechanical.  (A) 
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